Skip to content
This repository has been archived by the owner on Sep 30, 2024. It is now read-only.

writeManyInstances: improve error message if we run out of placeholders. #1265

Merged
merged 1 commit into from
Nov 17, 2020

Conversation

sjmudd
Copy link
Collaborator

@sjmudd sjmudd commented Nov 16, 2020

Description

If orchestrator uses prepared statements and uses the bulk insert options it can trigger a large bulk insert statement to be triggered which exceeds the 64k placeholders that are the limit in the MySQL server.

Related issue:

I have bumped into this on several occasions and the client side workarounds while not good have to be taken. This can also adversely affect the performance of keeping a cluster up to date.

To help the orchestrator administrator understand how to resolve this I have added code to recognise the specific error and to provide some more details of why the error happened, suggesting a configuration change.

Something like:

2020-11-16T15:11:37.295707+01:00 orchestrator orchestrator[57507]: flushInstanceWriteBuffer last_seen: writeManyInstances(?,true,true): error: Error 1390: Prepared statement contains too many placeholders, len(instances): 999
, len(args): 99999.  Reduce InstanceWriteBufferSize to avoid len(args) being > 64k, a limit in the MySQL source code.

Reducing the size of InstanceWriteBufferSize or alternatively setting: "MySQLInterpolateParams": true, would work.

@sjmudd
Copy link
Collaborator Author

sjmudd commented Nov 16, 2020

I see a system test failing but don't see/understand what's actually broken.

@shlomi-noach
Copy link
Collaborator

I see a system test failing but don't see/understand what's actually broken.

Seems like a flaky test which I should fix.

@shlomi-noach shlomi-noach merged commit 97eb9c2 into openark:master Nov 17, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants